feat: live score - #13
Merged
Merged
Conversation
/score takes a repository URL and returns the full score — signals, per-model breakdown, suggestions — for repos the leaderboard has never indexed. Indexed repos redirect to their canonical /repo/:id page. A Vercel function has no git binary, so the live path materializes the tree from the host API instead of cloning: every path present, real bytes only for the ~14 files a signal reads. Both paths then run the same scoreRepo(), and scripts/parity-check.ts asserts they agree — a signal that starts reading a new file would otherwise score as empty on the live path only, with no error and a wrong number on a public page. Reading searchParams for ?model= makes the route dynamic, so a segment-level revalidate would never cache the render. The score is cached explicitly instead, keyed by repo rather than model. Throws are not cached, so a rate limit or host blip reaches error.tsx rather than being pinned on a repo for the hour. GitLab and Bitbucket are implemented and score identically to a clone, but ship behind "support coming": GitLab paginates its tree at 100 entries and Bitbucket allows 60 unauthenticated requests an hour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
/changelog has always carried this and nobody goes looking for it; the version pill is a fact with no story attached. The notice renders CHANGELOG[0].title and nothing else, so there is no second copy of the release description to drift. The seen marker stores the version rather than a boolean, which is what makes the next release show again with no reset step or expiry logic. It is written at the moment of display, not on dismiss, so "exactly once per release" holds for a visitor who leaves after two seconds. Anchored under the nav link for the page the release is about, measured from that link's own rect — the nav's contents decide where it lands and they change. Below md the header collapses to a hamburger, so there is nothing to point at and nothing renders; the marker is deliberately not written in that case, or the one announcement would be spent on a screen that never showed it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ject `bun run seed` scores 350+ repos and kept every tree, which had grown to 36 GB locally. Nothing downstream reads the tree again — the score is already in memory by then. The workspace also moves from ./tmp-clones to the OS temp dir, because a clone inside the project is part of the Next.js module graph: one repo with a symlink loop (stripe/ai has one) fails `next build` with a Turbopack panic that names none of this. The dynamic join/readdirSync calls in the scorer make the tracer give up and pull the whole project into every function bundle. The scorer can't be annotated away — it is vendored verbatim into both sibling repos — so the weight is trimmed with outputFileTracingExcludes instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tsc does not see what only the bundler sees — a server-only import pulled into a client component, an RSC boundary violation. Without this the first place a broken build shows up is the Vercel deploy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
data/rank.db ships inside the deployment, so these responses cannot change until the next deploy. Uncached, every caller re-serialised the table — /api/repos returns the whole leaderboard. Matches the headers the badge and package routes already set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Weighted toward the thin sections rather than JS/TS: Swift and Dart gain two each, PHP two, GitLab two. gitlab.com/kicad/code/kicad is a subgroup path, which exercises the nested-namespace branch of parseRepoUrl that only gitlab-org/* covered. Each verified against its host API as public, non-fork, non-archived and not renamed — the criteria audit-seeds.ts enforces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Agent Friendly Code — score diffOverall: 91.0 → 91.0 (±0.0) Per model
No signals changed between base and head. Posted by agent-friendly-action. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Motivation
Ships tasks/0.7.0/ — cuts release 0.7.0.
▎ Goal (03-live-score-pages.md): Paste a public repo URL, get its score. Same numbers as the leaderboard, for repos the leaderboard has never seen, with nothing stored anywhere.
▎ Goal (01-tree-materializer.md): Given a public repo URL, produce a directory on disk that scoreRepo() scores identically to git clone — without a git binary, in ~1.5s for a typical repo.
The dashboard could only answer "how agent-friendly is a repo we already indexed?". That is the wrong question for anyone arriving with their own repo in mind.
Changes
Live scoring — lib/live-score/ materializes a scoreable directory from the host tree API (a Vercel function has no git binary): every path present, real bytes only for the ~14 files a signal reads. Both paths then run the untouched scoreRepo(). scripts/parity-check.ts + .github/workflows/parity.yml assert they agree, on a PR subset and nightly on the full fixture set.
Caching — reading searchParams for ?model= makes the route dynamic, so a segment-level revalidate never applies. The score is cached explicitly, keyed by repo not model. Throws are never cached, so a rate limit reaches error.tsx instead of pinning a failure on a repo for the hour.
Release notice — components/ReleaseAnnouncement.tsx, driven by CHANGELOG[0] so there is no second copy to drift. Anchored under the nav link for the page the release is about; not shown at hamburger widths, and deliberately not marked seen there either.
Fixes found along the way — bun run score now deletes each clone (36 GB had accumulated) and works outside the project, since a clone with a symlink loop breaks next build; CI now runs next build, which it never did; DB-backed API routes get cache headers.
Not shipped — GitLab and Bitbucket are implemented and score identically to a clone, but sit behind "support coming": GitLab paginates its tree at 100 entries, Bitbucket allows 60 unauthenticated requests/hour.
Testing
Docs + roadmap sync
Ships a roadmap item — removed from lib/roadmap.ts (0.7.0 → 0.8.0), added to lib/changelog.ts under the 0.7.0 bucket. AGENTS.md and README.md updated for the new directories, the parity-check command, the live-score threat surface, and a new client-side-persistence convention; CONTRIBUTING.md documents the parity workflow. The code-review and quality-check skills' JSON-LD allowlists were stale and are now in step with AGENTS.md.
Risks / rollback
fail for everyone at once. Tokenized it is 5,000/hour.
noindex, and a slug-shape guard rejects probes before any fetch. Per-IP rate limiting needs shared state and is deferred until abuse actually appears.